The use of dietary supplements is widespread, with many individuals taking them without a physician’s guidance. However, dietary supplements can potentially interact with both over-the-counter and prescription medications, and many of these interactions are unknown to patients. The documentation of dietary supplement use in structured medical records is often missing, but unstructured clinical notes contain additional information about supplement use. To take advantage of the clinical notes, we developed an NLP system to detect dietary supplement use in clinical notes and used survey data from a group of 377 patients from three healthcare facilities for evaluation. In particular, we applied the trained model to notes belonging to patients that had completed a survey evaluating their supplement use and evaluated the correlations between patients’ self-reported supplement use and that detected with NLP techniques in the clinical notes. We trained an SVM model on bigrams from snippets of text containing supplement keywords, resulting in an overall high performance (F1 = 0.914). The NLP results’ agreement with patient self-reported surveys, however, was variable, with good agreement in many cases (e.g., calcium, F1 = 0.83) and low agreement in others (e.g., folic acid, F1 = 0.39).
Implication: Health record systems generally contain medication lists; however, dietary supplements are not routinely tracked via this mechanism. Some dietary supplements have been shown, and others are strongly suspected, of having interactions with other medications. The good news is that we have demonstrated that NLP of semi-structured and unstructured clinical notes can reliably detect the use of many dietary supplements. The not-so-good news is that the agreement between the self-reported supplement use and the NLP results is not consistently high.
Our study has several strengths, including the ability to capture dietary supplement use from free-text clinical notes, which could enable future clinical studies on drug interactions and outcomes research. Moreover, we show that patients from multiple healthcare facilities often self-reported supplement use that contradicted what was recorded in the clinical record, indicating the importance of improving the documentation of supplement use in medical records. Finally, the use of a nationwide electronic health record system allowed for the generalizability of our findings across different healthcare facilities.
Limitations: Despite the promising results of this study, a limitation we encountered was the variability in performance between different supplements when compared to patient self-report in surveys. In error-checking our process, we manually reviewed cases where the SVM prediction did not match the survey answers. This uncovered multiple cases where the patient reported that they were not taking a specific supplement; however, the supplement was recorded in their medication list. There was an apparent pattern indicating that patients did not regard something as a supplement if it was prescribed by a physician. This may be the underlying cause of the lower agreement of our model in cases such as folic acid and melatonin, both of which are commonly prescribed by a physician. Future studies could address the mental model patients have of what they consider to be a supplement. In addition, roughly half of the supplements we studied did not have the number of observations needed for reliable results to be measured when compared to surveys. A larger survey set, or a more directed survey, would be useful to address this. Another limitation is that we did not perform feature selection or parameter optimization before training the SVM, nor did we compare the SVM against other machine learning algorithms. Although SVMs perform their own internal feature selection, better optimization may be obtained from feature selection and parameter tuning.
Another challenge we face in the study of dietary supplements is the lack of dosage, duration, and an exact start date for supplement use. While we were able to identify numerous mentions of the use of dietary supplements, the documentation of dosage, duration, and exact start date is inconsistent, if available at all. Unfortunately, this is a documentation problem that NLP cannot solve. With increasing awareness of the importance of holistic care for patients, we expect the documentation of dietary supplements to improve.
In order to further pursue this important topic, future work will possibly include a larger study with more data and survey participants. Additionally, future work will explore other machine learning algorithms, including the application of large language models such as those produced by bidirectional encoder representations from transformers (BERT), to improve NLP performance.
